Keyhole State Space Construction with Applications to User Modeling

Most models that explain observations in time depend on a structured state space as a basis for their modeling. We present methods to derive such a state space and its dynamics automatically from the observations, without any knowledge of their meaning or source. First, we build an explicit state space from an observation history in an off-line fashion (OFESI) by starting with the space induced by the observations and splitting a state when new substates significantly improve the information content of the state’s action distribution. We form these new substates by grouping fixed length histories leading up to the state. We apply our algorithm to the user modeling case and show that we can automatically build a meaningful stochastic dynamic model of application use. Second, we discuss prediction of the next observation and show that an approach based on on-line implicit state identification (ONISI) from observed history outperforms other prediction methods. Again, we are interested in user modeling, where we can predict future user actions better than another current algorithm. Both algorithms work without knowledge or modification of the application in use. Third, we apply our explicit state identification algorithm to the problem of state identification for Hidden Markov Models (SIHMM). Taking into account both observation and transition probabilities we learn structures for Hidden Markov Models in a few iterations.